Challenges in interpreting multi-modal data from trials and simulations

The Operational Research Team (ORT) is integrated at the Canadian Forces Maritime Warfare Centre (CFMWC) to provide an in situ analytical function, as part of the Centre’s overall vision “to deliver valid and effective maritime warfare tactics and maritime force development advice” [1]. One of the tasks assigned to the ORT by the CFMWC Commanding Officer (CO) is to better integrate the analysis of results produced by the Maritime Modelling and Simulation Cell (MMSC) and those produced by the Operational Analysis sections for each battlespace. Doing so requires understanding some fundamental limitations of analyzing limited amounts of data. This letter addresses concerns related to the particular challenges of dealing with multi-modal data, i.e., with data that has more than one peak in its distribution, generated at in-water trials and/or simulations. This can be of a particular concern if the data set is small—as it normally must be for in-water trials due to the cost—and thus more complex methods of analyzing multi-modal data sets would not work, and the modes may not even be well defined. In order to illustrate the difficulties of interpreting this type of data while enabling wide distribution, and thus hopefully wide discussion and understanding of the issue, notional data was generated for the purposes of this letter. However, it has been designed to resemble a case seen in real in-water and computer-based models; the details of these observations will be reported on via appropriate means.